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Abstract —Preference learning (PL) is a core area of machine learning 
that handles datasets with ordinal relations. As the number of generated 
data of ordinal nature is increasing, the importance and role of the PL 
field becomes central within machine learning research and practice. 
This paper introduces an open source, scalable, efficient and acces¬ 
sible preference learning toolbox that supports the key phases of the 
data training process incorporating various popular data preprocessing, 
feature selection and preference learning methods. 
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1 Introduction 

Preferences define a central concept in human decision 
making and have, thus, recently inspired the devel¬ 
opment of preference-centered computational systems. 
Preference data has been generated from a increasing 
number of disciplines that investigate preference han¬ 
dling and preference learning[3 (PL) including artificial 
intelligence and machine learning (ML), decision sup¬ 
port systems, marketing research, operations research, 
economics and human computer interaction. Given the 
increasing importance and availability of preference 
data, the several benefits of data collection protocols 
based on preferences and ranks [Ij and the advantages 
of PL over any other machine learning methods when 
ordinal data is treated |!2|, preference learning (|3| nowa¬ 
days defines one the key machine learning subareas. To 
facilitate the broad use of PL algorithms and methods 
among ML researchers, students, educators and practi¬ 
tioners this paper introduces the open source Preference 
Learning Toolbox (PLXfl 

While efficient PL applications already exist they focus 
only on a single PL method, thereby constraining the 
breadth of available PL algorithms. The large variety 
of classificafion methods available in the WEKA toolkit 
P4J are applicable to ordinal regression tasks through 
the framework LPCforSOS by Fiirnkran^l; however, this 
framework cannof frain models from partial orders given 
as comparisons or non-absolute ranks (e.g., as those 
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found in human preference reports, where measurement 
scales are subjective and vary across annotators and 
time) as it requires specialized PL methods not imple¬ 
mented in WEKA. In addition, that framework, as most 
PL applications, offers only a command-line interface 
making its use cumbersome without considerable prior 
knowledge about the PL paradigm, the particular meth¬ 
ods and the tool itself. 

PLT, instead, aims to provide an efficient collection of 
specialized PL mefhods but, most importantly, a rmified 
and accessible user interface for fhem. Beyond all, PLT 
offers a cross-platform preference learning framework 
for machine learning research in order fo supporf experf 
ML users but also users unfamiliar with PL methods. 
The toolbox includes in-line help with detailed and 
comprehensive documentation of the algorithms imple¬ 
mented and their adjustable parameters, and references 
fo the methods used. PLT implements the key phases of a 
machine learning or data mining process including data 
preprocessing, automatic feature selection and model 
training. Finally, PLT is available under the GNU Lesser 
General Public License and offers an accessible back¬ 
end to allow for a straightforward addition of new algo¬ 
rithms and methods at all phases of preference learning. 

2 A Toolbox for Preference Learning: 
Specifications and Key Modules 

The functionality offered by PLT allows users to 
generate computational models that map a set of 
input features to an ordinal output. The process of 
creating a model with PLT consists of a sequence of 5 
phases, each encapsulated within distinct modules to 
facilitate the extension of the tool. For the same reason, 
a graphical-user interface for all these phases has been 
implemented as separate modules decoupled from the 
functionality. PLT is written in Java 7 and the graphical 
interface is realized using JavaFX (requires Java SE 7ul3 
or later) which makes the tool available for a large 
number of Mac OS X, Linux and Windows distributions. 

Dataset Loading and Parsing: All preference learning 
techniques strive to create predictive preference models 
via empirical data. Naturally, the first phase of PL 
process involves the task of loading a dataset. In 
preference learning, models are constructed from a set 
of objects and the order (preferences) among them. 
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PLT accepts data in single-file and dual-file format. The 
single format supports datasets in which a total order 
among the objects is known (typically given as ranks 
or ratings) while the dual format is tailored for datasets 
with partial orders (given as comparisons between two 
or more objects). 

Users are informed about the validity of the dataset 
files they have provided through UI notifications, and 
provide several parser parameters (e.g. symbol used 
to separate entries and possibility of skipping lines) to 
ease the process of loading diverse datasets. Currently, 
PLT features five example datasets users can experiment 
with: three synthetic datasets with known preference 
functions (linear, quadratic and non-linear), the popular 
Sushi preference dataset |5j and the Maze-Ball affective 
preferences dataset j6j. 

Dataset Preprocessing: When a ML method is coupled 
with a particular dataset a data processing phase 
is usually required prior to the learning phase. PLT 
contains functionality to preview the object features 
within a loaded dataset, to select or omit particular 
features from the dataset, to alter feature representations 
(nominal to a binary representation and numeric to 
nominal) and to normalize the values of particular 
features (currently min-max and z-score normalization 
methods are supported). 

Automatic Feature Selection: While training can 
involve all available features, removing those which 
are not relevant to the sought predictive model is 
considered a good ML practice. In PLT, any features 
considered irrelevant can be deactivated manually. 
However, manually identifying these features is not 
always a trivial task while exhaustively exploring 
all possible feature subsets is not computationally 
plausible in datasets that contain many features. In such 
circumstances, users can opt to use an automatic feature 
selection (PS) algorithm. PLT currently supports two 
popular FS mechanisms, namely N-best individuals and 
sequential forward feature selection. Feature subsets are 
scored using the prediction accuracy of a model trained 
on the data using the PL methods described on the next 
phase. 

Preference Learning Methods: Once the dataset has 
been loaded and features have been preprocessed and 
selected, the next step is to train the model via PL. PLT 
enables the selection of fhe algorithm and its parameters 
as well as the validation mechanism to be used. 
Currently, PLT supports the ranking support vector 
machine method (Ranking S VM) introduced by Joachims 
12] and two artificial neural network (ANN) training 
algorithms: backpropagation and neuroevolution. PLT's 
Ranking SVM module was constructed on a LIBSVM 
foundation jH]. PL performance can be assessed via 
k-fold cross validafion and on the training set. 

The current version of PLT focuses on mefhods that 


train PL models from pairwise preferences derived 
from total or partial orders. Note that any totally 
or partially ordered set can be represented as a set of 
pairwise preferences without any loss of information ||3l. 

Reporting: During experiment execution a console 
window provides the user with status updates. 
After each completed learning trial PLT displays a 
configuration summary for each phase, a section 
showing the accuracy values of generafed models and 
an average method accuracy. Trained PL models can be 
selected and stored in JSON format. 

3 Benchmark Testing 

We evaluate the tool on synthetic and real datasets to 
demonstrate its efficiency and correctness. We utilise 
three synthetic datasets to compare the prediction accu¬ 
racies and CPU times of the methods in PLT against the 
tool IZ]. These datasets contain 10000 pairs of 
10-feature objects that are generated using a uniform ran¬ 
dom generator; a preference is generated for each pair, 
and the order is determined using a function of the object 
features. Each dataset uses a different function: linear, 
quadratic and non-linear (an artificial neural network 
with a hidden layer), respectively. Algorithm parameters 
were systematically tuned and the highest prediction 
accuracies are reported. The results are evaluated using 
3-fold cross-validation to keep a low number of runs. 

This test (see Table [ij yielded equivalent validation ac¬ 
curacies but significant differences in CPU times. While 
the linear SVM in is more than twice as 

fast as in PLT, using a radial basis function kernel (for 
the non-linear dataset) or a polynomial kernel (for the 
quadratic dataset) yielded faster training processes in 
PLT. Backpropagation managed to lower the CPU times 
in all experiments, while neuroevvolution (as expected) 
tends to have a longer training time. 

In addition to the tests on S 5 mthetic data, we also 
evaluate the correctness of the methods in PLT using 
the sushi preference dataset described in f9]. This dataset 
contains 1000 different objects with 18 features and 5000 
partial orders, each with a rank of 10 objects. Following 
the experimental protocol described in ]9j, we evaluate 
the methods using 5-fold cross validation and the Spear¬ 
man rank correlation coefficient. The values achieved by 
the PLT methods (0.377,0.373 and 0.363 for backpropaga¬ 
tion, neuroevolution and SVM, respectively) are within 
the range of performances ([0.370,0.405]) reported by f9j, 
further indicating the correctness of the implementation. 

4 Conclusions and Future Work 

This paper introduced the Preference Learning Toolbox, 
a user-friendly application for modeling ordinal rela- 
fions. The tool supports the key phases of any machine 
learning process offering preprocessing and automafic 
feafure selection frmctionalities. Currently, the toolbox 
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Linear preferences 

Quadratic preferences 

Non-linear preferences 

PLT-SVM 

PLT-BP 

PLT-NE 

94.41% (00:02:48) 
94.94% (00:00:13) 
92.15% (00:00:51) 

72.15% (00:08:30) 
84.30% (00:01:30) 
85.23% (01:44:11) 

85.99% (00:07:35) 
84.08% (00:01:24) 
85.87% (02:00:38) 


94.82% (00:01:33) 

77.60% (00:46:10) 

87.99% (00:31:03) 


TABLE 1 

Validation accuracy and CPU time of S'VM™”'" and the PLT methods (support vector machines, backpropagation 

and neuroevolution) on synthetic datasets. 


allows users to train SVM and artificial neural net¬ 
work models (via both backpropagation and artificial 
evolution) on input features selected with local search 
algorithms. Nevertheless, the tool has been designed 
with extensibility in mind and future work will focus on 
including other PL methods such as Bayesian networks 
ITOl as well as developing PL versions of other popular 
methods such as hidden Markov-models. 

PLT is in active development and strives to support 
the needs of users and developers alike. The software 
website provides the former with easy access to the latest 
distribution and updates, and allow them to suggest 
improvements and new functionalities. The same site 
offers a wiki, bug reporting capabilities and version 
control (git) for developers interested to partake in the 
improvement of the tool. 
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